Autonomous Helicopter Tracking and Localization Using a Self-Calibrating Camera Array

This paper describes an algorithm that tracks and localizes a helicopter using a ground-based trinocular camera array. Using background differencing and a Kalman filter, the helicopter is found in each of the camera images. The location of the moving helicopter in each image is then used to self-calibrate the relative positions and orientations of each of the cameras in the array while simultaneously estimating the 3-D trajectory of the helicopter with respect to the array. Once the camera array’s extrinsic parameters have been extracted, simple triangulation can be used in subsequent runs to identify the location of the helicopter in a camera coordinate frame. INTRODUCTION Position estimation is of critical importance in autonomous robotics research as it is the principal measurement used in machine control and localizing collected data. The approach in this project involves using three cameras located on the ground to track and localize a helicopter, such as the Stanford Autonomous Helicopter (Figure 1), in a fixed coordinate frame. The purpose is to replace an on-board GPS system to lighten the vehicle, make it robust to GPS occlusions, and to allow for more aggressive flight maneuvers. Figure 1: Stanford Autonomous Helicopter The cameras used by the system are located on the ground in positions that will cover a volume of air containing the space the helicopter will operate in. Because the rotation and translation relationship between each camera is unknown, this extrinsic data will need to be extracted through self-calibration of the array. Once the extrinsic data has been determined, then the x-y-z location of the aerial vehicle can be accurately and robustly tracked. This extrinsic information is usually obtained via calibration of the cameras in the scene utilizing a calibration object, such as a cube with a checkerboard pattern or the cameras are known to be in fixed locations and orientations with known extrinsic parameters. This is not ideal in a field environment because the above methods would require a recalibration of the cameras with the calibration aid every time a camera is jostled or would require a large structure that would fix the cameras in relation to each other while providing enough coverage to view the entire scene. Thus, the process of camera selfcalibration is crucial to the tracking problem. Through this the camera array will be able to estimates its geometry while deployed in the field without requiring modifications to the scene or the helicopter. This process also allows us to re-compute the calibration parameters on the fly if a camera has been moved. Without needing an explicit calibration target, self-calibration also serves to make the estimation methods partially invariant to the structure of the vehicle. Our approach uses multiple observations of the same scene motion to recover the extrinsic relationships between the cameras. In particular, this is done using a variant of the structure from motion (SFM) solution. SFM algorithms typically use camera motion to recover static scene structure; however, reversing this approach allows for the computation of the static camera geometry from scene motion This approach, which has only recently become feasible due to advances in desktop computing and imaging technology, is a novel approach for robotic localization. However, there are several related localization approaches in the field. Approaches like GPS and radar provide high precision localization accuracy, but tend to be expensive, hard to relocate,